Papers with cross-modal alignment knowledge
Improving Cross-modal Alignment for Text-Guided Image Inpainting (2023.eacl-main)
Copied to clipboard
| Challenge: | Existing methods allocate most of computation to visual encoding, while light computation on modeling modality interactions. |
| Approach: | They propose a novel model for text-guided image inpainting by improving cross-modal alignment knowledge by using a vision-language encoder and an image generator. |
| Outcome: | The proposed model achieves state-of-the-art performance compared with other strong competitors on two vision-language datasets. |
Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on temporal sentence grounding rely on expensive video-query paired annotations . despite this, there are no ground-truth annotations in the current work . |
| Approach: | They propose to use paired video-query and segment boundary annotations to generate temporal sentence grounding without training. |
| Outcome: | The proposed model outperforms existing unsupervised methods and beats supervised ones on two challenging datasets. |